Demo #12 - DAUF
Datadriven Analys och Uppföljning av KTHs Forskning
2025-12-05
Agenda
- Status update for the DAUF project
- New ABM 2025 with overall result and updates
- New beta version of subject based KTH Research Information app
- News related to data curation - new version of DiVA coming
- OpenAlex on Sunet
- Future directions and your questions and feedback
About the DAUF project
- Creating services and tools for presentation of research information data, improved data flows and connecting data sources within KTH
- Agile model with 2 week sprints
- Collaboration between KTH Library, RSO and ITA
- Part of IT portfolio for Research (Delportfölj forskning), within in the object “Publicering och analys”
Status and progress update
Progress overview - since last demo
This years version of ABM was released about a week ago
Recently released beta version of topics based KTH Research Information
POC for the KTH Indicators dashboard based on consolidated indicators collected from across KTH.
Tests and prep for GDP 2.0 (Gemensamma dataprojektet) - new standard for Swedish project data
Work to use OpenAlex to update DiVA, and to construct bibliometric database
Annual Bibliometric Monitoring 2025
Changes in ABM 2025
- More interactive graphs (plotly)
- Changed OA graph
- Enabled selection of number of rows for co-publication tables
- Some cosmetic changes
Brief ABM results for KTH
- Number of publications seems to have stabilized
- Citations indicators relatively stable
- Journal indicators stable but slightly increasing over last 5 years
- Small changes in co-publication patterns
- Share of Open Access publications sharply decreasing last year
- reasons unclear at the moment
KTH Research Information - Topics (beta)
- Selection based on Web of Science Citation Topics
- Three levels - macro, meso and micro
- Otherwise similar to KTH Research Information - Units
Data Curation - overview
- Preparations under way for migration to a future new version of DiVA
- Launch of new DiVA with API is postponed, and has new timeline
- All records need review to meet new (not yet finalized) requirements
- Updates are required for KTH curation tools and processes
- Broader discussions relating to a future data lake for “KTH Works”
- Publication data mirrored in a separate system under control from KTH
- Ability to cross reference other research outputs and auxiliary data from external sources
- Revamp curation process - increase automation and data enrichment from external sources, sync to DiVA repository
Data Curation and data flows
Object storage (S3)
- An on-premise data lake is powered by S3 storage using “minio”
- Data is curated and pushed using automated processes to https://data.bibliometrics.lib.kth.se (“minio” / S3)
- Applications and data consumers can then read (and/or write) data there
- See descriptions here for available data
General Dataflow
+--------------------------------+ | | | Data Sources | | | +--------------------------------+ | Clean / Crosscheck / Transform v +--------------------------------+ | | | Curated Data | | | +--------------------------------+ | Write / POST v +--------------------------------+ | | | [S3] Bronze/Silver/Gold | | | +--------------------------------+ | Read / GET v +--------------------------------+ | | | Data Consumer / Client | | | +--------------------------------+
DiVA curation
The DAUF project now harvests DiVA publication data from KTH using the OAI-PMH protocol which regularly updates duckdb databases, openly available from object storage:
- https://data.bibliometrics.lib.kth.se/kthcorpus/oai.db (Raw MODS in XML and JSON)
- https://data.bibliometrics.lib.kth.se/kthcorpus/diva.db (Tables / RDBMs format)
The database is regularly updated. This is WIP and jocularly codenamed “KaTHarsis”
- Harvest of KTH works in DiVA now available as relational database
- Ambition to decouple importing and curation from DiVA in preparation for new DiVA
- Can curate and annotate works using this database, aka “stoplists”
- Preparations to use APIs to sync data between DiVA repository and this database
DiVA curation stats
Journal articles in DiVA 2015 - 2025
y art_n_pi pi art_n_r r shr pct
2015 932 ░░░░ 879 ░░░░ ████████░░░░░░░ 51 %
2016 1142 ░░░░░ 1260 ░░░░░░ ███████░░░░░░░░ 48 %
2017 1922 ░░░░░░░░░ 855 ░░░░ ██████████░░░░░ 69 %
2018 2363 ░░░░░░░░░░░ 890 ░░░░ ███████████░░░░ 73 %
2019 3129 ░░░░░░░░░░░░░░░ 1418 ░░░░░░░ ██████████░░░░░ 69 %
2020 3240 ░░░░░░░░░░░░░░░ 880 ░░░░ ████████████░░░ 79 %
2021 2617 ░░░░░░░░░░░░ 1167 ░░░░░ ██████████░░░░░ 69 %
2022 2354 ░░░░░░░░░░░ 1302 ░░░░░░ ██████████░░░░░ 64 %
2023 3110 ░░░░░░░░░░░░░░░ 1264 ░░░░░░ ███████████░░░░ 71 %
2024 2456 ░░░░░░░░░░░░ 931 ░░░░ ███████████░░░░ 73 %
2025 2387 ░░░░░░░░░░░ 799 ░░░░ ███████████░░░░ 75 %
DiVA curation stats …
Conference papers in DiVA 2015 - 2025
y con_n_pi pi con_n_r r shr pct
2015 454 ░░░░░ 923 ░░░░░░░░░ █████░░░░░░░░░░ 33 %
2016 675 ░░░░░░░ 743 ░░░░░░░ ███████░░░░░░░░ 48 %
2017 757 ░░░░░░░░ 828 ░░░░░░░░ ███████░░░░░░░░ 48 %
2018 761 ░░░░░░░░ 679 ░░░░░░░ ████████░░░░░░░ 53 %
2019 908 ░░░░░░░░░ 898 ░░░░░░░░░ ████████░░░░░░░ 50 %
2020 804 ░░░░░░░░ 659 ░░░░░░░ ████████░░░░░░░ 55 %
2021 779 ░░░░░░░░ 670 ░░░░░░░ ████████░░░░░░░ 54 %
2022 781 ░░░░░░░░ 602 ░░░░░░ ████████░░░░░░░ 56 %
2023 1230 ░░░░░░░░░░░░ 750 ░░░░░░░░ █████████░░░░░░ 62 %
2024 1174 ░░░░░░░░░░░░ 329 ░░░ ████████████░░░ 78 %
2025 756 ░░░░░░░░ 456 ░░░░░ █████████░░░░░░ 62 %
DiVA curation stats …
Journal articles in 2025, by month
t art_n_pi pi art_n_r r shr pct
2025-01 236 ░░░░ 63 ░ ████████████░░░ 79 %
2025-02 207 ░░░░ 40 ░ █████████████░░ 84 %
2025-03 198 ░░░░ 31 ░ █████████████░░ 86 %
2025-04 247 ░░░░░ 34 ░ █████████████░░ 88 %
2025-05 160 ░░░ 66 ░ ███████████░░░░ 71 %
2025-06 196 ░░░░ 119 ░░ █████████░░░░░░ 62 %
2025-07 787 ░░░░░░░░░░░░░░░ 44 ░ ██████████████░ 95 %
2025-08 229 ░░░░ 55 ░ ████████████░░░ 81 %
2025-09 143 ░░░ 156 ░░░ ███████░░░░░░░░ 48 %
2025-10 100 ░░ 110 ░░ ███████░░░░░░░░ 48 %
2025-11 18 113 ░░ ██░░░░░░░░░░░░░ 14 %
DiVA curation stats …
Conference papers in 2025, by month
t con_n_pi pi con_n_r r shr pct
2025-01 148 ░░░░░░░░░░░ 39 ░░░ ████████████░░░ 79 %
2025-02 81 ░░░░░░ 14 ░ █████████████░░ 85 %
2025-03 92 ░░░░░░░ 35 ░░░ ███████████░░░░ 72 %
2025-04 90 ░░░░░░░ 31 ░░ ███████████░░░░ 74 %
2025-05 48 ░░░░ 21 ░░ ███████████░░░░ 70 %
2025-06 22 ░░ 52 ░░░░ █████░░░░░░░░░░ 30 %
2025-07 111 ░░░░░░░░ 85 ░░░░░░ █████████░░░░░░ 57 %
2025-08 39 ░░░ 59 ░░░░ ██████░░░░░░░░░ 40 %
2025-09 57 ░░░░ 65 ░░░░░ ███████░░░░░░░░ 47 %
2025-10 41 ░░░ 45 ░░░ ███████░░░░░░░░ 48 %
2025-11 29 ░░ 16 ░ ██████████░░░░░ 64 %
Swedish bibliometric resource (OpenAlex)
Demetrius
GDP
GDP (Gemensamma data för projekt) is an effort of a number of Swedish research funders to create a common data model for project data. The five funding agencies Energimyndigheten, Formas, Forte, Vetenskapsrådet and Vinnova is developing a standard which enables sharing of open data about fundings and related information.
The standard is developed in cooperation with a reference group including universities and other organisations within the university sector, KTH is a participant in the reference group.
GDP data mobilization
Version 2 of the GDP API has recently been released. Tooling has been developed at KTH for harvesting data from the different sources using this version of the API, see https://github.com/KTH-Library/gdp.
Regular harvesting of data from the API is now available from object storage “minio” at KTH: https://data.bibliometrics.lib.kth.se/projects/gdp/gdp.db
Future work and discussion
Future work and directions
- x
Related activities
- KTH CRIS/RIMS
- KTH Insights / datastyrning (MS Fabric/Power BI)
Questions and Answers
Please provide your input in chat or verbally.
- Questions, suggestions or comments?
If you prefer to give your feedback later or come up with questions after this demo, you are always welcome to email us at biblioteket@kth.se.
Thank you for attending!
DAUF - Demo 11 - 2025-12-05